Question Types#

Question Implementation

Each question type is implemented as a class the class also specifies the HTML/js templates to use for that question type. The constructor documents the logging variables that can be passed. See more on

All questions must have certain parameters:

  • question_text

  • question_id

the question_text can include markdown formatting which will be rendered with the markdown package

Normal Curve Questions#

This question has two normal curves, one moves and one does not.

Example#

Settings#

Use figure_type: NormalCurveSlider with the following parameters in figure_values

  • static_name : string legend text for static curve

  • static_color : hex including # hex code for the color to use for static curve, including a # sign as the first character

  • static_mean : number location of the static curve

  • static_curve_width : number width of curve, as the scipy.norm scale

  • dynamic_name : string legend text for dynamic curve

  • dynamic_color : hex including # hex code for the color to use for dynamic curve, including a # sign as the first character

  • dynamic_starting_mean : number the location where the slider starts

  • curve_width : number width of curves

  • num_slider_locs : integer number of slilder locations

  • min_slider_value : number the minimum value for the slider

  • max_slider_value : number the maximum value for the slider

  • overlap_decimals : integer number of place values to round the % overlap value to for both display and reporting, positive to the right of the decimal, negative for left of decimal (eg -2 rounds to nearest 100)

  • mean_decimals : integer number of place values to round the mean (position) value to for both display and reporting positive to the right of the decimal, negative for left of decimal (eg -2 rounds to nearest 100)

  • xaxis_title : string text label for the x axis

Tradeoff Questions#

this question type trades off between two two extremes over a number of models in the middle

Both formats accept the same dataset files, where each row represents one bar heigh for one slider location. The values should be compatible with deisplay in the figures, for example converting .74 to 75 to display percentages. There may be extra columns that are not used.

An example (that is used in the example plots):

alpha variable value metric group model_number percent
0 0.00 acc_0 0.747873 accuracy Black 0 74.787268
1 0.02 acc_0 0.748818 accuracy Black 1 74.881815
2 0.04 acc_0 0.749764 accuracy Black 2 74.976363
3 0.06 acc_0 0.750709 accuracy Black 3 75.070911
4 0.08 acc_0 0.751655 accuracy Black 4 75.165459

Line Graph Tradeoff#

This uses lines to show all metrics on one set of axes.

Example#

Settings#

Use figure_type: TradeoffLine with the following parameters for use in figure_values:

  • pretty_data_file : string file name of a tidy (tall) dataset with pretty content. that is any data transformations should occur on the data (eg scaling .7523943 to 75.23943 and expanding column names) column names can still rely on python conventions, before display the _ will be converted to space

  • slider_column : string name of column to use for the slider

  • slider_label : string name to display when labeling the slider postion values (and in hovertext)

  • x_col,y_col : string name of column to use for the xor y axis

  • trace_value1,trace_value2 : same as the values of x_col in the data file first,second value to filter (left, right metric)

  • trace1_hover, trace2_hover : string noun versions to use in the hovertext

  • y_min,y_max : numerical minimum and maximum values to fix the plot axies, if none, allow plotly to decide

  • num_digits : num digits to display color_col: string name of colum to use for the colring of the lines

  • color_hover : string noun to use for groups

  • disable_zoom : bool disable the zoom on the generated plot

  • anchor_name : string name for vertical bar default_selection :int model that is selected when laoding

Bar Graph Tradeoff#

This uses a set of bar graphs.

Example#

Settings#

Use figure_type: TradeoffBar with the following parameters for use in figure_values

  • pretty_data_file : string file name of a tidy (tall) dataset with pretty content. that is any data transformations should occur on the data (eg scaling .7523943 to 75.23943 and expanding column names) column names can still rely on python conventions, before display the _ will be converted to space

  • slider_column : string name of column to use for the slider

  • slider_label : string name to display when labeling the slider postion values (and in hovertext)

  • x_col,y_col : string name of column to use for the xor y axis

  • x_value1,x_value2 : same as the values of x_col in the data file first,second value to filter (left, right metric)

  • x_value1_hover, x_value2_hover : string noun versions to use in the hovertext

  • y_min,y_max : numerical minimum and maximum values to fix the plot axies, if none, allow plotly to decide

  • num_digits : num digits to display color_col: string name of colum to use for the colring of the bars color_hover: hover text to use for groups created by color

  • disable_zoom : bool disable the zoom on the generated plot default_selection :int model that is selected when laoding